Scaling Limits for the Transient Phase of Local Metropolis-Hastings Algorithms

This paper considers high-dimensional Metropolis and Langevin algorithms in their initial transient phase. In stationarity, these algorithms are well-understood and it is now well-known how to scale their proposal distribution variances. For the random walk Metropolis algorithm, convergence during the transient phase is extremely regular to the extent that the algorithm’s sample path actually resembles a deterministic trajectory. In contrast, the Langevin algorithm with variance scaled to be optimal for stationarity, performs rather erratically. We give weak convergence results which explain both of these types of behaviour, and give practical guidance on implementation based on our theory.